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^ (57) Abstract: A projector (1) includes a template memory (5), the template memory storing beforehand, for example data of tem- 
plate images made of only ruled lines. A displaying unit (6) of the projector (1) projects template images to a screen (S). A digital 

Q camera unit (8) carries out imaging of characters, etc., that are written on the screen (S), together with projected template images. 
A CPU (2) sends data of the image that is carried out imaging, to a PC (100). The PC (100) receives the image data, and stores it to 

^ a storing unit (103). 
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DESCRIPTION 

PROJECTION DEVICE, PROJECTION SYSTEM, 
AND IMAGE OBTAINMENT METHOD 



5 

Technical Field 

The present invention relates to a projection device, a projection system, 
and an image obtainment method. 

10 Background Art 

Conventionally, documents stored in personal computers as data, namely, 
documents made of characters and charts, etc., are projected expanded on a screen 
by a projector, in a presentation for describing projects and merchandise. There 
are projectors having a structure of converting to projection light, image 
15 information of documents output from the personal computer, as an image signal, 
using image converters such as liquid crystal and Micro-Mirror Array. 

In recent years, due to the miniaturization of projectors, meetings are 
. often carried out using projectors. In that case, by using a white board as a 
screen for projection, comments and underline can be directly written down on the 
20 white board by a marker, and charts can be created on the whiteboard so as 
organize a discussion. As this kind of projector, a presentation system using a 
camera installed in the projector is disclosed in Japanese Patent Application No. 
HI 1-305335, FIG.l. 

In a case where charts are created on a whiteboard during meetings, the 
25 below methods can be thought of. 

(1) Ruled lines are drawn by a marker, and characters, etc., in a frame are 
also written in by a marker, to a whiteboard. 
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(2) Ruled lines created by a personal computer is projected by a projector, 
and characters, etc., in a frame are written in by a marker, to a 
whiteboard. 

(3) Ruled lines created by a personal computer is projected by a projector, 
5 and characters, etc., in a frame are written in by input operation of the 

personal computer. 

However, with method (1), it takes time to draw the ruled lines. With 
method (2), it is necessary to start up a special application software to draw the 
ruled lines using the personal computer. With method (3), a plurality of 
10 participants surround the whiteboard and create charts, etc., holding markers, and 
thus can not participate in the discussion, and workability of directly writing 
down a chart on the screen becomes low. 

The present invention has been made in consideration of the above, and 
the object of the present invention is to provide a projection device and an image 
15 obtainment method that is possible to elevate workability. 

Disclosure of Invention 
A projection device according to a first aspect of the present invention, 
projects images on a screen (S), and comprises: 
20 a projection unit (6) whiclrprojects images on the screen based 

on provided image data; 

storing units (5, 21) which store data for generating template 
images that have shapes set beforehand, and 

a control unit (2) which obtains data for generating template 
25 images (Tl, T2, T3) from the storing units (5, 21), provides the generated image 
data based on the obtained data to said projection unit (6), and projects the 
template images to said projection unit (6). 
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By this kind of structure, workability of directly writing down a chart on 
the screen can be elevated. 

A projection system according to a second aspect of the present invention, 
projects images on a screen (S), and comprises: 
5 projection devices (1,51) which project the images on the screen 

(S); 

an image storing device (100) which stores data of images that 
are projected to the screen (S); wherein 

the projecting devices (1,51) comprise: 
10 a projection unit (6) which projects images based on the 

provided image data to the screen (S); 

storing units (5, 21) which store data of template images 
(Tl, T2, T3) where the shapes are pre-set; 

an imaging unit (8) which carries out imaging of the 

15 screen (S), and 

a sending unit (9) which sends data; and 
the image storing device (100) comprises: 

a storing unit (103) which stores data of document 
images that are projected to the screen (S), and 
20 a control unit (101) which extracts data of the document 

image from the storing unit (103), sends the extracted image data to the projection 
devices (1, 51), receives data of image sent from the projection devices (1, 51), 
and stores data corresponding it to data of the document image to the storing unit 
(103). 

25 An image obtainment method according to a third aspect of the present 

invention projects images on a screen, and comprises: 

a step of projecting a template image where a shape is pre-set, to 
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the screen, and 

a step of carrying out imaging of the screen where the template 
image is projected. 

5 Brief Description of Drawings 

FIG. 1 is a diagram showing a structure of a projection device shown in the first 
embodiment of the present invention. 

FIG. 2 is a block diagram showing a structure of a projector and a personal 
computer according to the first embodiment of the present invention. 
10 FIG. 3 is a diagram showing a line-up template as an example of a template image 
stored in a template memory of the projector. 

FIG. 4 is a diagram showing a conference note proceeding as an example of a 
template image stored in the same template memory. 

FIG. 5 is a diagram showing a global map template as an example of a template 
15 image stored in the same template memory. 

FIG. 6 is a flowchart showing operation of the projector in the first embodiment 
of the present invention. 

FIG. 7 is a diagram showing a structure of a projection device shown in the 
second embodiment of the present invention. 
20 FIG. 8 is a block diagram showing a structure of a projector and a personal 
computer according to the second embodiment of the present invention. 
FIG. 9A is a diagram showing ruled line data, which is template data that is stored 
in the memory card of the projector. 

FIG. 9B is a diagram showing cell data, which is template data that is stored in the 
25 memory card of the projector. 

FIG. 10 is a flowchart showing operation of the projector shown in the second 
embodiment of the present invention. 
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FIG. 1 1 is a flowchart showing content of ruled line transferring processing by the 
projector. 

FIG. 12 is a flowchart showing content of designation position determination 
processing by the projector. 
5 FIG. 13 is a flowchart showing content of ruled line attribute changing processing 
by the projector. 

FIG. 14 is a flowchart showing content of frame attribute changing processing by 
the projector. 

FIG. 15 is a flowchart showing content of ruled line adding processing by the 
10 projector. 

Best Mode for Carrying Out the Invention 
Embodiments of the present invention will be described with reference to 
the drawings. 

15 

First Embodiment 
The projection system of this embodiment, projects template images, 
such as images with only ruled lines, on a screen, and reads characters, etc., which 
are handwritten in the projected template image on the screen, by imaging. 
20 The projection system, comprises a projector 1 and a personal computer 

(PC) 2, which are connected to each other via an RGB cable 200 and a USB cable 
300. 

The projector 1 is for projecting images of charts during presentations, 
etc., and has a built-in digital camera for carrying out imaging of the screen S, 
25 such as a white board, etc. 

The projector 1 comprises a projection unit la, and an imaging unit lb, at 
the front side of the body. The projection unit la is for projecting display 
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images, and includes optical systems, such as projection lenses for projecting 
display images. The imaging unit lb includes optical systems, such as imaging 
lenses for carrying out imaging of the screen S. 

As shown in FIG. 2, the projector 1 comprises a CPU 2, a ROM 3, a 
5 RAM 4, a template memory 5, a display unit 6, a key input unit 7, and a digital 
camera unit 8. 

The template memory 5 is for storing data for generating template images. 
Template images are images, where the shape is pre-set. The template memory 5 
stores various data for generating template images, such as a lineup template Tl 
10 shown in FIG. 3, a conference note proceeding template T2 shown in FIG. 4, and 
a global map template T3 shown in FIG. 5. 

The display unit 6 is for projecting document images of documents 
(documents having characters and charts, etc.) that are output from the PC 2, and 
template images, on the screen S. The display unit 6 includes a light source, 
15 image converters, a drive circuit, and an optical system (Not shown). 

The light source emits light, such as a krypton light. The image 
converters are for converting the light of the light source to a projection light, and 
comprises liquid crystal elements and Micro-Mirror Array, etc. 

The drive circuit activates the image converters, in response to RBG 
20 signals input from an input terminal 18. 

The optical systems include imaging lenses, etc., at the projection unit la. 
The key input unit 7 comprises an operation button for displaying 
template images, and an operation button for stopping the display of the template 
images. The key input unit 7 comprises an imaging button for carrying out 
25 imaging of the screen S as an object, by the digital camera unit 8. The key 
input unit 7 sends operation information to the CPU 2, when these buttons are 
pressed by the user. 
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The key input unit 7 comprises an operation button, etc., for designating 
attributes. 

The digital camera unit 8 carries out imaging of the screen S as the object, 
by the imaging button of a key input unit 15 being pressed. A digital camera 
5 unit 16 comprises an optical system, an image pickup device, and an image data 
signal compression processing circuit (Not shown). The optical system includes 
imaging lenses, etc., at the imaging unit lb. The image pickup device is for 
photoelectrical transferring an optical image focused by the optical system to an 
image signal, and comprises a CMOS sensor and a CCD, etc. The image data 

10 signal compression processing circuit compresses digital signals that are 
digitalized, being output from the image pickup device. 

The ROM 3 stores programs. The CPU 2 controls each above described 
unit that describe the RAM 4 as an operation memory, in accordance with the 
program stored in the ROM 3. Concretely, the CPU 2 controls the display unit 6 

15 to project the document image on the screen S. When operation information 
indicating that the operation button for displaying the template is pressed, is sent 
from the key input unit 7, the CPU 2 receives this operation information from the 
key input unit 7 as a projection instruction, and projects the template image on the 
screen S. When operation information indicating that the imaging button is 

20 pressed, is sent from the key input unit 7, the CPU 2 receives the operation 

information as imaging instructions, and controls the digital camera unit 8 to carry 
out imaging of the screen 8, in accordance with the operation information. By 
this, the CPU 2 obtains an image that includes recorded information recorded in 
the documents and the screen S, and outputs the obtained image to the PCI 00. 

25 The projector 1 comprises an image input terminal 9, and a USB terminal 

10. 

The RGB cable 3 is connected to the image input terminal 9. The image 
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input terminal 9 is a terminal for being provided the RBG signal from the PC 100, 

j 

and is provided at an input image processing circuit, which is not shown. The 
* input image processing circuit includes an A/D converter, etc. 

The USB cable 300 is connected to the USB terminal 10. The USB 
5 terminal 10 is for sending image data obtained by the projector 1 carrying out 
imaging, and is provided at a communication interface circuit for USB, not shown 
in the drawings, which is connected to the CPU 2. 

The PC 100 provides data of documents that are to be projected to the 
screen S, to the projector 1, and comprises a CPU 101, a RAM 102, a storing 
10 device 103, an input device 104, and a display device 105. 

The input device 104 comprises a plurality of keys, and a mouse, etc., 
which is connected to the body. When the user operates the input device 104, 
the input device 104 sends operation information indicating the operation content 
of the user, to the CPU 101. 
15 The display device 105 displays document images read from the storing 

device, and displays images sent from the projector 1, and comprises a LCD, and 
the drive circuit, etc., thereof. 

The storing device 103 comprises a hard disk, etc, having a relatively 
large storage capacity, and stores a presentation program for creating and editing 
20 documents for presentation. The storing device 103 stores data of documents 
made up of characters and charts, for projecting to the screen S, as a document 
file. 

The CPU 101 controls each part of the PC 100, in accordance with the 
program stored in the storing device 103. The CPU 101 uses the RAM 102 as an 
25 operation memory. When the operation information indicating that data of a 
document is to be sent, is sent from the input device 104, the CPU 101 sends data 
of the document to the projector 1 . When the image data is sent from the 
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projector 1, the CPU 101 stores correlating the image data sent from the projector 
1, and sent document data, at the storing unit 103. 

The PC 100 comprises an output terminal 106 for RBG signals, and a 
USB terminal 107. The RGB cable 200 is connected to the output terminal 106. 
5 The output terminal 106 is provided at the image signal processing circuit, not 
shown in the drawings, which is connected to the CPU 101. The USB cable 300 
is connected to the USB terminal 107. The USB terminal 107 is provided at the 
communication interface circuit for USB, not shown in the drawings, which is 
connected to the CPU 101. 
10 Next, operation of the projection system according to the present 

embodiment will be described with reference to the flowchart shown in FIG. 6. 
First, operations of the PC 100 will be described. 

When the user designates data of a document operating the input device 
104, the input device 104 of the PC 100 sends the operation information of the 
15 user to the CPU 101. 

The PC 100 reads data of documents designated by the user, from the 
storing device 103, and displays a document image of a predetermined page to the 
display device 105, in accordance with the operation information of the user,. 
The PC 100 outputs data of the document image displayed to the display device 
20 104, via the RGB cable 200, to the projector 1 (Step SA1). 

The PC 100 determines whether the image data is sent from the projector 
1 or not (Step SA2). The PC 100 constantly executes the determination by 
interrupt handling. 

In a case where it is determined that the image data is not sent from the 
25 projector 1, (Step SA2: YES), the PC 100 displays an image based on the image 
data sent from the projector 1, to the display device 105. The PC 100 stores the 
image data sent from the projector 1, correlating it with data of the document 
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image, to the storing device 103 (Step SA3). 

Next, the operation of the projector 1 will be described. 
The projector 1 projects document images on the screen S, based on data 
of the document image output from the PC 100 (Step SB1). 
5 The projector 1 determines whether the user operated the operation 

button for template display, based on the operation information from the key input 
unit 7 (Step SB2). 

In a case where it is determined that the operation button for template 
display is not operated by the user (Step SB2: NO), the projector 1 continuously 
10 projects document images on the screen S (STEP SB1). 

In a case where it is determined that the operation button for template 
display is operated by the user (STEP SB2: YES), the projector 1 receives the 
operation information as a projection instruction, and reads data of the template 
image that correspond to the operated button, from the template memory 5 (Step 
15 SB3). 

The projector 1 switches the projection image from the document image 
to the template image based on the read data, and projects the template image to 
the screen S (STEP SB4). 

The projector 1 determines whether imaging operation is carried out by 
20 the user, based on the operation information from the key input unit 7 (Step SB5). 

In a case where the imaging button of the key input unit 7 is not pressed, 
the projector 1 determines that imaging operation has not been carried out (Step 
SB5: NO). In this case, the projector 1 determines whether imaging of template 
image has completed or not, based on the operation information from the key 
25 input unit 7 (Step SB8). 

In a case where the operation button for stopping the template display is 
not pressed, the projector 1 determines that imaging of the template image has not 
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been completed (Step SB8: NO). In this case, the projector 1 re-determines 
whether imaging operation is carried out or not (Step SB 5). 

Here, if the imaging button of the key input unit 7 is pressed, the 
projector 1 determines that imaging operation is carried out (Step SB5: YES). In 
5 this case, the projector 1 receives the operation information as an imaging 
instruction, controls the digital camera unit 8, and carries out imaging of the 
screen S where the template image is projected (Step B6). 

The projector 1 outputs the image data obtained by carrying out imaging, 
to the PC 100, via the USB cable 300 (Step SB7). 

10 The projector 1 re-determines whether imaging of the template image is 

completed or not (Step SB8), and in a case where it is determined that imaging of 
the template image has not been completed (Step SB8: NO), the processing of 
Steps SB5 to SB7 are repeatedly carried out. In this way, the projector 1 outputs 
image data to the PC 100 each time imaging operation is carried out. 

15 In a case where the operation button for stopping the template display is 

pressed, the projector 1 determines that imaging of the template image is 
completed (Step SB8: YES). In this case, the projector 1 switches the image 
from the template image to the original document image, and projects the 
document image on the screen S (Step SB9). 

20 This operation will be described more concretely. 

In a case where a plurality of participants carry out a meeting, etc., 
surrounding the screen S, the participants press the operation button that the key 
input unit 7 of the projector 1 comprises, when a conference note proceeding 
template T2, such as shown in FIG. 4 is to be used. When the operation button 

25 for displaying the conference note proceeding template T2 is pressed, the 
projector 1 responds to the operation information, and reads data of the 
conference note proceeding template T2 shown in FIG. 4, from the template 
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memory 5 (Process of Step SB2, SB3 in FIG. 6). The projector 1 projects this 
image on the screen S, instead of the projected document image (Process of SB4 
in FIG. 6). 

Next, after the participants record characters on the screen 2 with a 
5 marker, the imaging button of the key input unit 7 of the projector 1 is pressed. 
The projector 1 carries out imaging of the screen S, and sends the image data to 
the PC 100, in response to the imaging button being pressed (Process of Step SB5 
to SB 8 in FIG. 6). 

When the operation button for stopping the display of the template image 
10 is pressed, the projector 1 projects the original document image on the screen S, 
instead of the template image, in response (Process of Step SB9 in FIG. 6). 

The PC 100 stores image data sent from the projector 1, correlating it 
with the document, at the storing device 103 (Process of Step SA3). 

In a case where the PC 100 stores image data that is sent from the 
15 projector 1, at the storing device 103 (Step SA3), the PC 100 is structured so that 
it can identify which image data corresponds to which document data. For 
example, the PC 100 stores image data sent from the projector 1 in an image file. 
The PC 100 gives a part of the name of the data file of the document, to the file 
name of this image file, and stores the image file in the storing device 103. 
20 In a case where image data concerning the same document is sent a 

plurality of times, the PC 100 puts branch numbers, such as "-1", and "-2" at the 
end of the file name. 

As described above, in the first embodiment of the present invention, for 
example, a template image such as an image with only ruled lines, is projected to 
25 a screen S, and characters, etc, that are handwritten to the screen S, using the 
projected template image, is obtained by carrying out imaging. 

Therefore, it is not necessary for the user to write ruled lines by a marker 
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on the screen S (whiteboard) to make charts, etc., and time necessary to create 
charts can be saved. A plurality of participants can surround the screen S 
(whiteboard), and participate in a discussion, holding markers. As a result, work 
of creating a chart by directly writing down on the screen S, can be improved. 
5 It is not necessary to start a special application software by the PC 100, 

and template images can be projected immediately. 

Furthermore, if a plurality of kinds of template images are pre-stored as 
image data, various template images can be projected, in accordance with a 
variety of uses, and usability will improve. 
10 Because images obtained by projecting handwritten characters, etc., on 

the screen S, are stored in the storing device 103, handwritten information written 
on the screen S, can be used afterwards. 

In the present embodiment, because image data obtained by imaging, is 
stored correlated with documents, handwritten information can be easily used. 

15 

Second Embodiment 
Next, the second embodiment of present invention will be described. A 
projection system according to the second embodiment of the present invention is 
structured so that template images can be edited. 
20 As shown in FIG.7, the projection system according to the second 

embodiment of the present invention comprises a projector 51, a PC 100, and a 
laser pointer 400. The laser pointer 400 radiates a spot light P, and is for 
indicating a position on the screen S, by radiating the spot light P on the screen S. 
The laser pointer 400 is used for designating an editing position of the template 
25 image projected on the screen S. 

In the second embodiment, the template images are not stored in the 
template memory 5, but pre-stored in a memory card. The projection system 
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obtains template images from the memory card. 

As shown in FIG. 8, the projector 51 comprises a throttle (not shown in 
the drawings), and a card interface circuit (card I/F) 22. The throttle is for 
loading the memory card 21. The card I/F 22 is for inputting and outputting data 
5 between the memory card 21 . 

The projector 51 and the PC 100 are connected to each other only via the 
RGB cable 200. In the second embodiment, the USB terminal 10 and the 
communication interface circuit is omitted. 

The memory card 21 stores a plurality of template data. The template 
10 data is for generating template images, and includes ruled line data 21a, and cell 
data 21b. The ruled line data 21a is for generating ruled lines, and as shown in 
FIG. 9A, comprises coordinate data, line width data, color data, and line type 
data. 

The coordinate data indicates a start point and an endpoint of a plurality 
15 of longitudinal ruled lines and horizontal ruled lines, in accordance with the 
number of rows and number of columns. For example, the start point and the 
endpoint of ruled line Kl, shown in FIG. 9A, is respectively expressed by 
coordinate data (xl, yl), (x2, y2). The coordinate data is expressed by the 
number of pixels of the display unit 6 of the projector. For example, if the 
20 maximum pixel of the horizontal direction and the longitudinal direction is 

800x600, the maximum value of the coordinate data showing the start point and 
the endpoint of the ruled line is (800,600). 

The cell data 21b is for generating cells, and as shown in FIG. 9B, 
comprises coordinate data, line width data, color data, and line type data. 
25 The coordinate data shows the start point and the endpoint of the right to 

left, up and down ruled lines (Ksl to Ks4), that form each frame (SI to Sn) of the 
coordinate data. For example, the cell SI is formed by ruled lines Ksl to Ks4, 
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and the start point and endpoint of ruled line Ksl can be expressed by the 
coordinate data (xl, yl), (x2, y2). In the same way as the coordinate data of the 
ruled line data 21a, the coordinate data of the cell data 21b is expressed in the 
number of pixels of the display unit of the projector 51. 
5 The line width data shows the width of the ruled line that forms the cell. 

The color data shows the color inside of the cell. The line type data shows the 
type of ruled line that forms the cell. 

The key input unit 7 comprises various operation buttons. Various 
functions, such as selection function of template, display function, function for 
10 starting editing, and function for giving instructions for editing content, etc., are 
allotted to the operation buttons. 

The CPU 2 obtains information of a editing place that the laser pointer 
400 indicates, and specifies data corresponding to the editing place, based on the 
obtained information of the editing place. The CPU 2 obtains the specified data 
15 from the memory card 21, and edits the specified data, based on the operation 
information from the key input unit 7, which indicates the editing content. 

Next, operation of the projection system, according to the present 
embodiment, will be described. 

When the user operates the operation button of the key input unit 7, for 
20 displaying a template, while projecting a document, the projector 51 operates in 
accordance with the flowchart shown in FIG. 10, in response to the operation. 

When the operation button for template selection is pressed, the CPU 2 of 
the projector 51 obtains this operation information from the key input unit 7, and 
reads the template data of the selected template image from the card memory 21, 
25 based on the obtained operation information. The CPU 2 controls the display 
unit 6 to project the template image on the screen S (Step SCI), 

Concretely, when the CPU 2 reads template data from the card memory 
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21, the CPU 2 develops in RAM 4, image data of the template, based on the read 
template data. 

As described above, the template data comprises ruled line data 21a, and 
cell data 21b, such as shown in FIG. 9A, and FIG. 9B. The CPU 2 displays ruled 
5 line Kl from (xl, yl) to (x2, y2), based on the coordinate data of ruled line data 
21a. In the same way, the CPU 2 displays ruled lines K2 to Kn, based on the 
coordinate data of the ruled line data 21a. In this way, image data of the 
template is developed in RAM 4, and data of the template image is generated. 

The CPU provides data of the generated template image to the display 
10 unit 6. The display unit 6 projects the provided template image to the screen S. 

Next, the CPU 2 determines whether to start editing or not, based on the 
operation information from the key input unit 7 (Step SC2). 

In a case where the operation button for starting editing is not pressed, 
the CPU 2 determines that editing is not to start (Step SC2: NO). In this case, 
15 the CPU 2 waits until the operation button for starting editing is pressed. 

In a case where the operation button for stating editing is pressed, the 
CPU 2 determines that editing is to be started (Step SC2: YES), and determines 
the editing content, based on the operation information from the key input unit 7. 

First, the CPU 2 determines whether the editing is ruled line transferring, 
20 or not (Step SC3). Ruled line transferring processing is for transferring a ruled 
line to a designated direction for a designated amount. 

When the operation button for "ruled line transferring" is pressed, the 
CPU 2 determines that the editing content is ruled line transferring (Step SC3 : 
YES). In this case, the CPU 2 carries out ruled line transferring processing, 
25 according to a flowchart, which will be later described (Step SC4). 

After the ruled line transferring processing is completed, the CPU 2 
re-determines whether to start editing or not (Step SC2). 
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If the operation button for "ruled line transferring" is not pressed, after it 
is determined to start editing, the CPU 2 determines that the editing content is not 
ruled line transferring (Step SC5). Ruled line attribute changing processing is 
for changing the attribute of a ruled line, such as width of line, color, and line 
5 type. 

When the operation button for "ruled line attribute changing" is pressed, 
the CPU 2 determines that the editing content is ruled line attribute changing 
(Step SC5 : YES). In this case, the CPU 2 carries out ruled line attribute 
changing processing, according to a flowchart, which will be later described (Step 
10 SC6). 

After the ruled line attribute changing processing is completed, the CPU 
2 re-determines whether to start editing or not (Step SC2). 

If the operation button for "ruled line attribute changing" is not pressed, 
after it is determined to start editing, the CPU 2 determines that the editing 
15 content is not ruled line attribute changing (Step SC5 : NO). In this case, the 
CPU 2 determines whether the editing content is frame attribute changing or not 
(Step SC7). Frame attribute changing processing is for changing ruled line 
width that form the frame (cell), and coloring in the frame. 

When the operation button for "frame attribute changing" is pressed, the 
20 CPU 2 determines that the editing content is frame attribute changing (Step SC7 : 
YES). In this case, the CPU 2 carries out frame attribute changing processing, 
according to a flowchart, which will be later described (Step SC8). 

After the frame attribute changing processing is completed, the CPU 2 
re-determines whether to start editing or not (Step SC2). 
25 If the operation button for "frame attribute changing" is not pressed, the 

CPU2 determines that the editing content is not frame attribute changing (Step 
SC7 : NO). In this case, the CPU 2 determines that the editing content is ruled 
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line adding, and carries out ruled line adding processing, according to a flowchart, 
which will be later described (Step SC9). Ruled line adding processing is for 
adding ruled lines. 

After carrying out these processing, the CPU 2 ends display processing of ' 
5 the template image, when the operation button for ending display of the template 
image is pressed. 

Next, ruled line transferring processing carried out by the projector 51, in 
Step SC4 of FIG. 10, will be described, based on the flowchart shown in FIG. 11. 

The CPU 2 determines whether the operation button for starting editing is 
10 pressed or not, based on the operation information from the key input unit 7 (Step 
SD1). 

In a case where it is determined that the operation button for starting 

editing is not pressed (Step SD1 : NO), the CPU 2 waits until the operation button 

for starting editing is pressed. 
15 In a case where it is determined that the operation button for starting 

editing is pressed (Step SD1 : YES), the CPU 2 carries out designation position 

determining processing, according to the flowchart shown in FIG. 12 (Step SD2). 
First, the digital camera unit 8 is controlled by the CPU 2, and projects 

the screen S (Step SE1). A template image is projected on the screen S. 
20 The CPU 2 clips a target projection area of a template image, from the 

image obtained by carrying out imaging by the digital camera unit 8 (Step SE2). 

The CPU2 corrects trapezoid deformation of the clipped projection area 

of the image, and detects the part that has the most brightness in the corrected 

image area, as a radiation position of a spot light P (Step SE3). 
25 When the radiation position of the spot light P is determined, the CPU 2 

converts the radiation position of the spot light P to coordinate data for template 

data (Step SE4). 
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The CPU 2 specifies the ruled line designated by the spot light P, based 
on the coordinate that indicates the radiation position of the spot light P (Step 
SD3 in FIG. 11). 

The CPU 2 determines whether the operation button for instructing 
5 transferring direction is pressed or not, based on the operation information from 
the key input unit 7 (Step SD4). 

In a case where it is determined that the operation button for instructing 
transferring direction is not pressed (Step SD4 : NO), the CPU 2 waits until the 
operation button for instructing transferring direction is pressed. 
10 In a case where it is determined that the operation button for instructing 

transferring direction is pressed, (Step SD4 : YES), the CPU 2 transfers the 
designated ruled line to a designated direction, for a designated amount (Step 
SD5). Concretely, the CPU 2 changes coordinate data in the ruled line data 21a, 
shown in FIG. 9A, of the designated ruled line, based on the designated direction 
15 and amount. 

The CPU 2 changes data corresponding to the cell data 21b, shown in 
FIG. 9B, in accordance with coordinate data in the ruled line data 21a, shown in 
FIG. 9A. 

The CPU 2 generates a new template image, based on the changed data. 
20 The display unit 6 re-projects the generated template image to the screen S (Step 
SD6). 

The CPU 2 determines whether the operation button for ending editing is 
pressed, based on the operation information from the key input unit 7 (Step SD7). 
In a case where it is determined that the operation button for ending 
25 editing is not pressed (Step SD7 : NO), the CPU 2 once again carries out 
processing of steps SD4 to SD6. 

In a case where it is determined that the operation button for ending 
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editing is pressed (Step SD7 : YES), the CPU 2 ends the ruled line transferring 
processing. 

Next, ruled line attribute changing processing carried out by the projector 
51, in step SC6 in FIG. 10, will be described with reference to the flowchart 
5 shown in FIG. 13. 

The CPU 2 determines whether the operation button for starting editing is 
pressed or not, based on the operation information from the key input unit 7 (Step 
SF1). 

In a case where it is determined that the operation button for starting 
10 editing is not pressed (Step SF1 : NO), the CPU 2 waits until the operation button 
for starting editing is pressed. 

In a case where it is determined that the operation button for starting 
editing is pressed (Step SF1 : YES), the CPU 2 carries out designation position 
determination processing, according to the flowchart shown in FIG. 12 (Step 
15 SF2). 

The CPU 2 specifies a ruled line designated by the user, based on a 
coordinate indicating the radiation position of the spot light P (Step SF3). 

The CPU 2 determines whether attribute is designated or not, based on 
the operation information from the key input unit 7 (Step SF4). 
20 If the operation button for attribute designation is not pressed, the CPU 2 

determines that the attribute has not been designated yet (Step SF4 : NO). In this 
case, the CPU 2 waits until the attribute is designated. 

If the operation button for attribute designation is pressed, the CPU 2 
determines that the attribute is designated (Step SF4 : YES), and determines 
25 whether the attribute to be changed is "width" of ruled line or not, based on the 
operation information from the key input unit 7 (Step SF5). 

If the operation button for designating "width" of ruled line, as the 
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attribute to be changed, is pressed, the CPU 2 determines that instruction for 
changing "width" of ruled line is carried out (Step SF5 : YES). In this case, the 
CPU 2 determines whether the operation button for selecting "width" of ruled line 
is pressed or not, based on the operation information from the key input unit 7 
5 (Step SF6). 

In a case where it is determined that the operation button for selecting 
"width" of ruled line has not been pressed (Step SF6 : NO), the CPU 2 waits until 
the operation button for selecting "width" of ruled line is pressed. 

In a case where it is determined that the operation button for selecting 
10 "width" of ruled line is selected (Step SF6 : YES), the CPU 2 changes the 

designated ruled line width to a selected width (Step SF7). Concretely, the CPU 
2 changes the width data of the designated ruled line in the ruled line data 21a 
shown in FIG. 9A. 

In accordance with the change in width data of the ruled line data 21a 
15 shown in FIG. 9 A, the CPU 2 changes data corresponding to the cell data 21b 
shown in FIG. 9B. 

The CPU 2 generates a new template image, based on the changed ruled 
line data 21a. The display unit 6 re-projects the generated template image on the 
screen S (Step SF13). 
20 Next, after the attribute is designated, in a case where it is determined 

that the operation button for selecting "width" of ruled line is not pressed (Step 
SF5 : NO), the CPU 2 determines whether the attribute to be changed is color of 
ruled line or not, based on the operation information from the key input unit 7 
(Step SF8). 

25 In a case where it is determined that the operation button designating 

"color" of ruled line is pressed (Step SF8 : YES), the CPU 2 determines that 
instructions for changing the "color" of ruled line is carried out, and determines 
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whether the operation button for selecting "color" of ruled line is pressed or not 
(Step SF9). In a case where it is determined that the operation button for 
selecting "color" of ruled line is not pressed (Step SF9 : NO), the CPU 2 waits 
until the operation button for selecting the "color" of ruled line is pressed. 
5 In a case where it is determined that the operation button for selecting the 

"color" of ruled line is selected (Step SF9 : YES), the CPU 2 changes the color of 
the designated color to a selected color (Step SF10). Concretely, the CPU 2 
changes color data of the designated ruled line in the ruled line data 21a shown in 
FIG. 9A. 

10 In accordance with the change of color data in the ruled line data 21a 

shown in FIG. 9A, the CPU 2 changes data that corresponds to cell data 21b 
shown in FIG. 9B. 

The CPU 2 generates a new template image, based on the changed data. 
The display unit 6 re-projects the generated template image on the screen S (STEP 

15 SF13). 

In a case where it is determined that the operation button for designating 
the "color" of ruled line is not pressed (Step SF8 : NO), the CPU 2 determines 
that instructions for changing "line type" of line is carried out. 

In this case, the CPU 2 determines whether the operation button for 

20 selecting "line type" of ruled line is pressed or not, based on the operation 
information from the key input unit 7 (Step SF1 1). In a case where it is 
determined that the operation button for selecting "line type" of ruled line is not 
pressed (Step SF1 1 : NO), the CPU 2 waits until the operation button for selecting 
the "line type" of ruled line is pressed. 

25 In a case where it is determined that the operation button for selecting 

"line type" is pressed (Step SF1 1 : YES), The CPU 2 changes the line type of the 
designated ruled line to a selected line type (Step SF12). Concretely, the CPU 2 
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changes the line type data of the designated ruled line, in the ruled line data 21a 
shown in FIG. 9A. 

In accordance with the change of line type data in the ruled line data 21a 
shown in FIG. 9 A, the CPU 2 changes data that corresponds to the cell data 21b 
5 shown in FIG. 9B. 

The CPU 2 generates a new template image, based on the changed data. 
The display unit 6 re-projects the generated template image on the screen S (Step 
SF13). 

The CPU 2 ends the ruled line attribute changing processing, after the 
10 template image is re-projected on the screen S. 

Next, frame attribute changing processing carried out by the projector 51, 
in Step SC8 in FIG. 10, will be described with reference to the flowchart shown in 
FIG. 14. 

The CPU 2 determines whether the operation button for starting editing is 
15 pressed or not, based on the operation information from the key input unit 7 (Step 
SGI). 

In a case where it is determined that the operation button for starting 
editing is not pressed (Step SGI : NO), the CPU 2 waits until the operation button 
for starting editing is pressed. 
20 In a case where it is determined that the operation button for starting 

editing is pressed (Step SGI : YES), the CPU 2 carries out designation position 
determining processing, according to the flow chart shown in FIG. 12 (Step SG2). 

The CPU 2 specifies a frame designated by the spot light P of the laser 
pointer 400, from the projected template image, based on the coordinate indicating 
25 the radiation position of the spot light P (Step SG3). 

The CPU 2 determines whether the attribute to be changed is designated 
or not, based on the operation information from the key input unit 7 (STEP SG4). 
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If the operation button for attribute designation is not pressed, the CPU 2 
determines that the attribute to be changed has not been designated yet (Step 
SG4 : NO). In this case, the CPU 2 waits until the attribute is designated. 

If the operation button for attribute designation is pressed, the CPU 2 
5 determines that the attribute to be changed has been designated (Step SG4 : YES). 
In this case, the CPU 2 determines whether instructions for changing "width" of 
the frame has been carried out or not (STEP SG5). 

If the operation button for designating "width" of the frame is pressed, 
the CPU 2 determines that instructions for changing the "width" of the frame that 
10 forms the frame, has been carried out (Step SG5 : YES). In this case, the CPU 2 
determines whether the operation button for selecting "width" of ruled line is 
pressed or not (Step SG6). 

In a case where it is determined that the operation button for selecting 
"width" of ruled line is not pressed (Step SG6 : NO), the CPU 2 waits until the 
15 operation button for selecting "width" of ruled line is pressed. 

In a case where it is determined that the operation button for selecting 
"width" of ruled line is pressed (Step SG6 : YES), the CPU 2 changes the width of 
the ruled line that forms the designated frame to a selected width (Step SG7). 
Concretely, the CPU 2 changes the width data of the up, down, right, and left 
20 ruled line that form the designated frame, in the cell data 21b shown in FIG. 9B. 

The CPU 2 generates a new template image, based on the changed data. 
The display unit 6 re-projects the generated template image to the screen S (Step 
SG10). 

Next, after the operation button for attribute designation is pressed, in a 
25 case where it is determined that the operation button for designating "width" of 
frame is not pressed (Step SG8 : YES), the CPU 2 determines that instructions for 
changing "color for coloring in" the frame is carried out. In this case, the CPU 2 
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determines whether the operation button for selecting the color for coloring in, is 
pressed or not (Step SG8). 

In a case where the CPU 2 determines that the operation button for 
selecting the color for coloring in, is not pressed (Step SG8 : NO), the CPU 2 
5 waits until the operation button for selecting the color for coloring in, is pressed. 

In a case where it is determined that the operation button for selecting the 
color for coloring in, is pressed (Step SG8 : YES), the CPU 2 changes the color 
for coloring in, in the designate frame, to a selected color (step SG9). 
Concretely, the CPU 2 changes the color data of the designated frame, in the ruled 
10 line data 21b shown in FIG. 9B. 

The CPU 2 generates a new template image based on the changed data. 
The display unit 6 re-projects the generated template image to the screen S (Step 
SG10). 

The CPU 2 ends the frame attribute change processing, after re-projecting 
15 the generated template image to the screen S. 

Next, ruled line adding processing carried out by the projector 51, in step 
S9 of FIG. 10, will be described with reference to the flowchart shown in FIG. 15. 

The CPU 2 determines whether start point instruction has been completed 
or not, based on the operation information from the key input unit 7 (Step SHI). 
20 If the operation button for instructing the start point is not pressed, the 

CPU 2 determines that start point indication has not been completed (Step SHI : 
NO). In this case, the CPU 2 waits until the operation button for instructing the 
start point is pressed. 

If the operation button instructing the start point is pressed, the CPU 2 
25 determines that start point instruction is completed (Step SHI : YES). In this 
case, the CPU 2 carries out designation position determination processing, 
according to the flowchart shown in FIG. 12 (Step SH2). Then, the CPU 2 stores 
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the obtained coordinate as the coordinate of the start point, to the RAM 4. 

The CPU 2 determines whether endpoint instruction has been completed 
or not, based on the operation information from the key input unit 7 (Step SH3). 

If the operation button for instructing endpoint is not pressed, the CPU 2 
5 determines that endpoint instruction has not been completed (Step SH3 : NO). 
In this case, the CPU 2 waits until the operation button for instructing endpoint is 
pressed. 

If the operation button for instructing endpoint is pressed, the CPU 2 
determines that endpoint instruction is completed (Step SH3 : YES). In this case, 
10 the CPU 2 carries out designation position determination processing, according to 
the flowchart shown in FIG. 12 (Step SH4). The CPU 2 stores the obtained 
coordinate as the coordinate of the endpoint, to the RAM 4. 

The CPU 2 adds new line data that includes the designated start point and 
endpoint to the ruled line data 21a of the projected template image (Step SH5). 
15 The CPU 2 adds new line data, also to the cell data 21b, in accordance with the 
addition of the ruled line data 21a. 

The CPU 2 generates a new template image, based on the changed data. 
The display unit 6 re -projects the generated template image to the screen S (Step 
SH6). 

20 The CPU 2 ends the ruled line adding processing, after re-projecting the 

template image to the screen S. 

As described above, the projector 51 of the present embodiment can edit 
template images. 

Therefore, in the same way as the first embodiment, an effect of saving 
25 time in creating charts is gained, and a requested template image can be easily 
obtained. 

Template images are retained as the above described template data (ruled 



WO 2004/057565 




•CT/JP2003/016272 



27 

line data 21a and cell data 21b), and the template image is projected based on the 
template data. Therefore, memory capacity necessary for saving template 
images can be made smaller, and more template images can be stored. 

Various embodiments are possible for carrying out the present invention, 
5 and is not limited to the above embodiments. 

For example, in the first embodiment, the operation button for displaying 
a template corresponds to the template image. However, the key input unit 7 
may comprise a switching button for switching template images, and the CPU 2 
may switch template images, everytime the switching button is pressed. By this 
10 kind of structure, selection of a requested template image can be carried out with 
fewer switching buttons. 

In the second embodiment, processing of "ruled line transferring", "ruled 
line attribute changing", and "frame attribute changing" are described as the 
editing processing. However, the editing processing is not limited to these. 
15 For example, editing processing, such as erasing, etc., ruled lines may be carried 
out. 

In the second embodiment, ruled lines and frames of the template image 
are selected one by one, to carry out various editing. However, ruled lines and 
frames selected as an editing target, is not limited to one. For example, by 

20 designating number of lines and number of rows, a plurality of ruled lines and 
frames can be designated. The entire template image may be selected, and the 
entire template image may be edited at once. 

In the second embodiment, a case where template data for generating 
template image in a chart form is stored in the memory card 21 is described. 

25 However, template data for generating template images made of only horizontal 
ruled lines and longitudinal ruled lines, namely only ruled line data 21a may be 
stored in the memory card 21. The data may be stored in another recording 
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medium, other than the memory card 21. 

■ In the second embodiment, the laser pointer 400 is used for designating 
specific ruled lines and frames. However, an operation button may be used 
instead of the laser pointer 400, and a structure where ruled lines and frames are 
5 designated by an operation button, is possible. In this case, the projector 51 does 
not have to comprise the digital camera unit 8. 

Editing processing may be carried out by various methods. For example, 
every time the operation button is pressed, ruled lines and frames that are editing 
choices may be switched. The switched ruled lines and frames may be expressed 
10 using color, and template images may be sequentially updated, and projected. 
Ruled lines and frames may be designated by an image such as a mouse pointer 
being displayed superposed on the template image, and moving it by an operation 
button which designates direction. 

In the second embodiment, editing of an existing template image is 
15 described. However, a new template image may be generated, by comprising a 
structure that has the user designate a number of ruled lines and a number of rows, 
and width of the ruled line, without using the existing template image. 

The projector 51 may store a storage button for designating storing to the 
memory card 21 of a newly created template image, after the template image is 
20 edited. 

The second embodiment may be structured in the same way as the first 
embodiment, so that the projector 51 outputs the image data taken by the digital 
- camera unit 8 to the PC 100, and the PC 100 stores the image data output from the 
projector 51 to the storing device 103. 
25 The digital camera unit 8 in the first embodiment and the second 

embodiment, are not limited to being built-in to the projector 1 and 51, and a 
general purpose digital camera may be used. In this case, if the projector 1 and 
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5 1 and the digital camera comprise for example a USB communication interface, 
data communication can be carried out by a USB cable, and if the projector 1 and 
51 and the digital camera comprise a radio communication interface, data 
communication by radio becomes possible. 
5 The patent application claims the Paris Convention Priority based on 

Japanese Patent Application No. 2002-369133 filed with the Japan Patent Office 
on December 20, 2002, the complete disclosure of which is hereby incorporated 
by reference. 
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CLAIMS 

1 . A projection device which projects images on a screen (S), 
comprising: 

5 a projection unit (6) which projects images on the screen based 

on provided image data; 

storing units (5, 21) which store data for generating template 
images that have shapes set beforehand, and 

a control unit (2) which obtains data for generating template 
10 images (Tl, T2, T3) from said storing units (5, 21), provides the generated image 
data based on the obtained data to said projection unit (6), and projects the 
template images to said projection unit (6). 

2. The projection device according to claim 1, wherein: 

said storing units (5, 21) store pixel information of said template 
15 images (Tl, T2, T3) as data for generating said template images (Tl, T2, T3); and 
said control unit (2) obtains the pixel pattern information from 
said storing units (5, 21), and generates said template images (Tl, T2, T3) to 
provide to the projection unit (6), based on the obtained pixel pattern information. 

3. The projection device according to claim 1, wherein: 

20 said storing units (5, 21) store template data (21a, 21b) for 

drawing ruled lines and generating said template images (Tl, T2, T3) , as data for 

generating said template images (Tl, T2, T3), and 

said control unit (2) obtains said template data (21a, 21b) from 

said storing units (5, 21), draws ruled lines based on the obtained template data 
25 (21a, 21b), and generates data of said template images (Tl, T2, T3) to be provided 

to said projection unit (6). 

4. The projection device according to claim 3, comprising: 
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an indication unit (400) which indicates editing position of said 
template images (Tl, T2, T3) projected on said screen (S), and 

an input unit (7) which inputs editing content of data that 
corresponds to said editing position, based on the obtained editing position, 
5 wherein said control unit (2) obtains information of editing 

position indicated by said indication unit (400), specifies data corresponding to 
said editing position based on the obtained editing position, obtains the specified 
data from the storing units (5, 21), and edits the obtained data based on the editing 
content that the input unit (7) input. 
10 5. The projection device according to claim 4, wherein: 

said storing units (5, 21) store ruled line data (21a) that define 
ruled lines that are to be drawn, as said template data, and 

said control unit (2) specifies ruled line data (21a) that 
corresponds to said editing position, based on the obtained editing position 
15 information, and obtains the specified ruled line data (21a) from the storing units 
(5,21). 

6. The projection device according to claim 5, wherein: 

said storing units (5, 21) store ruled line data (21a) including 
ruled line attribute information that indicates the attribute of the ruled line that is 
20 to be drawn, and 

said control unit (2) edits ruled line attribute information 
including said rule line data (21a), based on the editing content that said input unit 
(7) input. 

7. The projection device according to claim 4, wherein: 

25 said storing units (5, 21) store cell data (21b) that defines a cell 

that is surrounded by ruled lines that form said template images (Tl, T2, T3) as 
said template data, and 
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said control unit (2) specifies cell data (21b) that corresponds to 
said editing position, based on the obtained editing position information, and 
obtains the specified cell data (21b) from said storing units (5, 21). 

8. The projection device according to claim 7, wherein: 

5 said storing units (5, 21) store cell data (21b) that include cell 

attribute information indicating attribute of cells, and 

said control unit (2) obtains cell attribute information including 
said cell data (21b) from said storing units (5, 21), edits the obtained cell attribute 
information, based on the editing content that said input unit (7) input, and stores 
10 the edited cell data (21b) to said storing units (5, 21). 

9. The projection device according to claim 4, wherein: 

said indication unit (400) is for radiating spot light to said screen 
(S), and comprises an imaging unit (8) which carries out imaging of said screen 
(S), and 

15 said control unit (2) controls the imaging unit (8) to carry out 

imaging of the screen (S) where said template images (Tl, T2, T3) are projected, 
and said spot light is radiated, obtains position relationship of the spot light of 
said indication unit (400) and said template images (Tl, T2, T3) from the image 
obtained by said imaging unit (8) carrying out imaging, and obtains editing 

20 position information of said template images (Tl, T2, T3) based on the obtained 
position relationship. 

10. A projection device which projects images on a screen (S), 
comprising: 

a projection unit (6) which projects images based on provided 
25 image data to said screen (S); 

storing units (5, 21) which store data for generating template 
images that have shapes set beforehand; 
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an imaging unit (8) which carries out imaging of said screen (S); 
a command reception unit (7) which receives commands for 
controlling said projection unit (6) and said imaging unit (8), and 

a control unit (2) which provides data for generating template 
5 images (Tl, T2, T3) stored in said storing units (5, 21), to said projection unit (6), 
in accordance with a projection command that said command reception unit (7) 
received, projects said template images (Tl, T2, T3) to said projection unit (6), 
and controls said imaging unit (8) to carry out imaging of said screen (S), in 
accordance with an imaging command that said command reception unit (7) 
10 received. . . 

11. A projection system which projects images on a screen (S), 
comprising: 

projection devices (1,51) which projects the images on said 

screen (S); 

15 an image storing device (100) which stores data of images that 

are projected to said screen (S); wherein 

said projecting devices (1,51) comprise: 

a projection unit (6) which projects images based on the 
provided image data to said screen (S); 
20 storing units (5, 21) which store data of template images 

(Tl, T2, T3) where the shapes are pre-set; 

an imaging unit (8) which carries out imaging of said 

screen (S), and 

a sending unit (9) which sends data; and 
25 said image storing device (100) comprises: 

a storing unit (103) which stores data of document 
images that are projected to said screen (S), and 
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a control unit (101) which extracts data of said 
document image from said storing unit (103), sends the extracted image data to 
the projection devices (1,51), receives data of image sent from the projection 
devices (1, 51), and stores data corresponding it to data of said document image to 
5 said storing unit (103). 

12. An image obtainment method which projects images on a screen, 
comprising: 

a, step of projecting a template image, where a shape is pre-set to 

said screen, and 

10 a step of carrying out imaging of said screen, where said 

template image is projected. 

13. The image obtainment method according to claim 12, wherein 
said step of projecting said template image to said screen further comprises: 

a step of storing data of the template image to be projected to 
15 said screen beforehand, and 

a step of extracting said stored data of the template image, and 
projecting it to said screen. 
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